We present a new pre-trained language model (PLM) for modern Hebrew, termed AlephBERTGimmel, which employs a much larger vocabulary (128K items) than standard Hebrew PLMs before. We perform a contrastive analysis of this model against all previous Hebrew PLMs (mBERT, heBERT, AlephBERT) and assess the effects of larger vocabularies on task performance. Our experiments show that larger vocabularies lead to fewer splits, and that reducing splits is better for model performance, across different tasks. All in all this new model achieves new SOTA on all available Hebrew benchmarks, including Morphological Segmentation, POS Tagging, Full Morphological Analysis, NER, and Sentiment Analysis. Subsequently we advocate for PLMs that are larger not only in terms of number of layers or training data, but also in terms of their vocabulary. We release the new model publicly for unrestricted use.
translated by 谷歌翻译
虽然视觉和语言模型在视觉问题回答等任务上表现良好，但在基本的人类常识性推理技能方面，它们会挣扎。在这项工作中，我们介绍了Winogavil：在线游戏，以收集视觉和语言协会（例如，狼人到满月），用作评估最先进模型的动态基准。受欢迎的纸牌游戏代号的启发，Spymaster提供了与几个视觉候选者相关的文本提示，另一个玩家必须识别它们。人类玩家因创建对竞争对手AI模型而具有挑战性的联想而获得了回报，但仍然可以由其他人类玩家解决。我们使用游戏来收集3.5k实例，发现它们对人类的直观（> 90％的Jaccard索引），但对最先进的AI模型充满挑战，其中最佳模型（Vilt）的得分为52％ ，成功的位置在视觉上是显着的。我们的分析以及我们从玩家那里收集的反馈表明，收集的关联需要多种推理技能，包括一般知识，常识，抽象等。我们发布数据集，代码和交互式游戏，旨在允许未来的数据收集，可用于开发具有更好关联能力的模型。
translated by 谷歌翻译